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A SHORTENED STATUTORY PERIOD FOR REPLY IS SET TO EXPIRE 3 MONTH(S) OR THIRTY (30) DAYS, 
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1)13 Responsive to communication(s) filed on 10 January 2006 . 
2a)n This acKon is FINAL. 2b)l3 This action is non-final. 

3) D Since this application is in condition for allowance except for fonmal matters, prosecution as to the merits is 

closed in accordance with the practice under Ex parte Quayle, 1935 CD. 1 1 , 453 O.G. 213. 
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4) 0 Claim(s) is/are pending in the application. 
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Continued Examination Under 37 CFR 1.114 

A request for continued examination under 37 CFR 1.114, including the fee set forth in 
37 CFR 1 .17(e), was filed in this application after final rejection. Since this appUcation is 
eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1 . 1 7(e) 
has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 
37 CFR 1.1 14, Applicant's submission filed on 9/7/2005 has been entered. 

Allowable Subject Matter 

Claims 9, 18, and 27 objected to as being dependent upon a rejected base claim, but 
would be allowable if rewritten in independent form including all of the limitations of the base 
claim and any intervening claims. 

The following is a statement of reasons for the indication of allowable subject matter: 
Creating adjacency matrices representing smallest portions of the set ofmorphism graph that 
map the input and rule graphs to the alphabet graph using pre-images of parts of the alphabet 
graph marked for change; and multiplying the adjacency matrix associated with the input 
graph by a transpose of the adjacency matrix associated with the rule graph. 
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Claim Rejections - 35 USC§103 

The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set forth in 
section 102 of this title, if the differences between the subject matter sought to be patented and the prior art are 
such that the subject matter as a whole would have been obvious at the time the invention was made to a person 
having ordinary skill in the art to which said subject matter pertains. Patentability shall not be negatived by the 
manner in which the invention was made. 

Claims 1-8, 10-17, and 19-26 rejected under 35 U.S.C. 103(a) as being unpatentable over 
Uwe Assmann; Title: Graph rewrite systems for program optimization, ACM Transactions on 
Programming Languages and Systems (TOPLAS), Volume 22 Issue 4, July 2000, (refers as 
Assmann), and further in view of Euripides G.M. Petrakis, Member, IEEE, and Christos 
Faloutsos (hereafter refers as a "Euripides"). 

Claim 1. 

In regard to the preamble of the claim 1, Assmann on page 583 under the subject of 
introduction teaches how graph rewriting can be used for the specification and generation of 
program optimizations (Examiner's comment: it's obvious for an ordinary person skill in the art 
to acknowledge the graph rewrite system involves a computerized method). The central idea of 
method is to regard all of the information in an optimizer as a set of relational graphs. Program 
objects, intermediate code instructions, and predicates of the entities of the program are 
represented either as nodes or relations. Program analysis and transformation are performed 
using graph rewriting. Typically, analyzing programs means enlarging graphs with new edges, 



Application/Control Number: 10/044,168 Page 4 

Art Unit: 2672 

which represent the information, while code transformation means rewriting graphs by deleting 
and attaching sub graphs. The preamble is, as follows: "A computerized method of graph 
rewriting, a graph having nodes representing entities and edges between the nodes representing 
relationships between entities, the method comprising". For more information regarding a graph 
having nodes representing entities and edges between the nodes, see Assmann on page 587 xmder 
section "2". 

As claim invention discloses that comparing an input graph representing a description 
scheme for mukimedia content with a set of predefined template graphs, see Assmann on page 
628 at forth paragraph teaches the rule sets can be transformed with the magic set transformation 
which has the effect that tuples are preselected fi-om the database before joins are performed. The 
description scheme representing structure and semantics of the multimedia content, and the 
tenplate graphs representing model description schemes. Assmann on page 588 teaches a graph- 
morphism, (e.g., Morphisms are often depicted as arrows from their domain to their codomain, 
e.g. if a morphism/has domain^and codomain 7, it is denoted/ : X-^ Y). Assmann on page 
588 Unes 1-8 teaches a function that compares the source and target nodes for edges. Assmann 
on page 584 at third paragraph teaches that the graph is specified in a data model. Also on page 
587 teaches the graph nodes and edges must be modeled: The types of graph nodes. Among 
these are intermediate code instructions such as expressions, statements, loops, and procedures. 
Also, analyzed information can be encoded in nodes of graphs, e.g., variable definitions, or 
expression equivalence classes. The types of graph edges (relations). These represent all relevant 
predicates, which should be inferred from the intermediate representation. Some examples are 
relations such as classical flow dependencies, equivalence relations, calling relations, or control 
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flow relations. In addition, the information, which the frontend produces, must be modeled, e.g., 
expression tree pointering or statement lists. Obviously, this study represents structure and 
semantic of the multimedia content (Assmann in figs. 4-5 on page 590 illustrates the multimedia 
content). 

As claim invention discloses in the last Une of claim 1 that vaUdating the input graph 
when there is a match with a template graph. Assmann on page 591 in figs. 6-7 illustrates e.g., in 
fig. 6 the middle graph covers what the claim language claimed, as e.g., A: 1 at the left of the fig. 
6 matches A: 1 at the right side of the fig. 6. Similarly, in fig. 7 A:R1 matches A: 1 and A:R2 
matches A: 2, with d matches the relationship between the two nodes. 

However, Assmann does not show supporting for validating the input graph when there is 
a match with a template graph. Euripides on page 437 in second column teaches the first term in 
Eq. (1) is the cost of matching (validating as the claim language specifies) associated nodes, 
while the second term is the cost of matching the relationships between such nodes. In our 
setting, only a subset of the objects in the stored image S needs to be matched. There is no cost if 
the data image contains extra objects; however, we assume that the cost is infinite if the data 
image is missing one of the objects of the query. COST is the cost of matching features of 
objects or features of relationships between associated objects. The distance between images Q 
and S is defined as the minimum distance conputed over all possible mappings F(): see Eqs. 2-4. 
Similarity searching in an IDB of stored ARGs requires that all images within distance t must be 
retrieved. Specifically, we have to retrieve all the images S that satisfy the Eq. 4 condition. 

Thus, it would have been obvious to one of ordinary skill in the graphical writing system 
to support for multimedia content in fig. 1 with its corresponding ARG of Euripides for 



Application/Control Number: 1 0/044, 1 68 Page 6 

Art Unit: 2672 

searching by image content in image databases that represented by attributed relational graphs, 
because Assmann teaches this to be known in the graph rewrite system for program optimization. 

Claim 2. 

The computerized method of claim 1, wherein the comparing uses a graph matching 
process. Assmann in fig. 7 illustrates that matching process. Euripides on page 436 under section 
2. 1 discloses image descriptions are given in terms of object properties, and in terms of 
relationships between objects. The textbook approach to capture this information is the 
Attributed Relational Graphs (ARGs). In an ARG, graph nodes represent the objects, and arcs 
between such nodes represent the relationships between objects. Both nodes and arcs are labeled 
by attributes corresponding to properties (features) of objects and relationships, respectively. 

Claim 3. 

The computerized method of claim 2, wherein the conparing comprises: creating adjacency 
matrices representing the input graph and the set of template graphs. Assmann is silenced about 
creating adjacency matrices representing the input graph and the set of tenplate graphs. 
Euripides on page 440 under section 4. 1 and in second column teaches that additional features 
that could be used include the average gray-level and texture values, moments, or Fourier 
coefficients, etc., as object descriptors; relative size, amount of overlap-ping, or adjacency, etc., 
can be also used to characterize the relationships between objects. 
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Claims 4 and 5. 

The conputerized method of claim 1 further comprising: evaluating the input graph against a set 
of pre-defined alphabet graphs; and applying a rule associated with a matching alphabet graph to 
the input graph, the rule represented by a rule graph and a set of morphism graphs. Assmann 
does not explicitly specify the term "alphabet graph". Euripides on page 437 in column one 
discloses matching a query and a stored graph is treated as a sub graph isomorphism problem. 
Examiner comment: the meaning of morphism is an abstraction of a function or ma pping 
between two objects. 

Claim 6. 

The computerized method of claim 5, wherein the evaluating conprises: creating 
adjacency matrices for the input graph and the set of alphabet graphs. Euripides on page 439 in 
the first column discloses that the CAFIIR system proposes the "iconic index tree" to accelerate 
the search on facial unages. One novelty of the system is that it can process "fuzzy" (i.e., 
subjective or in-complete) queries, through the so-called "fuzzification" technique, which 
translates the feature space to a fuzzy space. Also see page 440 the last paragraph before section 
4.2 Euripides discloses set of alphabet graph or image translated or scaled with respect to each 
other. 

Claims 7 and 8. 

The computerized method of claim 4, wherein the applying comprises: performing 
pushout/puUback operations. As Applicant on page 1 1 paragraph 0029 discloses that the two 
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particular operations in algebraic graph grammars are suitable to build graph rewriting 
techniques for description scheme graphs: pushout and pullback. Pushouts and pullbacks can be 
thought of as sums and products, respectively. The limitations in these claims are obvious 
because in a graph rewrite systems for program optimization performs pullback and pushback 
operations to find the matching graph. Euripides on page 437 in Eqs. 1 -4 clearly shows the 
meaning of sums and products. Also in fig. lb illustrates the pullback (also called the fiber 
product) is the limit of a diagram consisting of at least two morphisms r23: v2 — ► v3 and rl2: vl 
— ►vS with a common codomain (A set within which the values of a function lie (as opposed to 
the range, which is the set of values that the fiinction actually takes). E?q)licitly, the pullback of 
the morphisms r23 and rl2 consists of an object vO and two morphisms rOl: vO — ► vl and r02: vO 
— ► v3 for which the diagram. The categorical dual of a pullback is a called a pushout . 

Claim 10. 

The rejection of claim 1 is similar to the rejection for claim 10. 
Claims 11-17 

The rejection of claims 2-8 is similar to the rejection of claims 11-17. 
Claims 19-26 

The rejection of claims 2-8 is similar to the rejection of claims 19-26. 
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Conclusion 

Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Javid A. Amini whose telephone number is 571-272-7654. The 
examiner can normally be reached on 8-4pni 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Kee Tung can be reached on 571-272-7794. The fax phone number for the 
organization where this application or proceeding is assigned is 571-273-8300. 

Information regarding the status of an appUcation may be obtained from the Patent 
Application Information Retrieval (PAIR) system. Status information for pubUshed appUcations 
may be obtained from either Private PAIR or PubUc PAIR. Status information for unpublished 
apphcations is available through Private PAIR only. For more information about the PAIR 
system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR 
system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). 

Javid A Amini 

Examiner 
Art Unit 2672 

Javid Amini 




